---
title: "Reference: Context Relevancy | Evals | Kastrax Docs"
description: Documentation for the Context Relevancy Metric, which evaluates the relevance of retrieved context in RAG pipelines.
---

# ContextRelevancyMetric ✅

The `ContextRelevancyMetric` class evaluates the quality of your RAG (Retrieval-Augmented Generation) pipeline's retriever by measuring how relevant the retrieved context is to the input query. It uses an LLM-based evaluation system that first extracts statements from the context and then assesses their relevance to the input.

## Basic Usage ✅

```typescript
import { openai } from "@ai-sdk/openai";
import { ContextRelevancyMetric } from "@kastrax/evals/llm";

// Configure the model for evaluation
const model = openai("gpt-4o-mini");

const metric = new ContextRelevancyMetric(model, {
  context: [
    "All data is encrypted at rest and in transit",
    "Two-factor authentication is mandatory",
    "The platform supports multiple languages",
    "Our offices are located in San Francisco"
  ]
});

const result = await metric.measure(
  "What are our product's security features?",
  "Our product uses encryption and requires 2FA.",
  );

console.log(result.score); // Score from 0-1
console.log(result.info.reason); // Explanation of the relevancy assessment
```

## Constructor Parameters ✅

<PropertiesTable
  content={[
    {
      name: "model",
      type: "LanguageModel",
      description: "Configuration for the model used to evaluate context relevancy",
      isOptional: false,
    },
    {
      name: "options",
      type: "ContextRelevancyMetricOptions",
      description: "Configuration options for the metric",
      isOptional: false,
    }
  ]}
/>

### ContextRelevancyMetricOptions

<PropertiesTable
  content={[
    {
      name: "scale",
      type: "number",
      description: "Maximum score value",
      isOptional: true,
      defaultValue: "1",
    },
    {
      name: "context",
      type: "string[]",
      description: "Array of retrieved context documents used to generate the response",
      isOptional: false,
    }
  ]}
/>

## measure() Parameters ✅

<PropertiesTable
  content={[
    {
      name: "input",
      type: "string",
      description: "The original query or prompt",
      isOptional: false,
    },
    {
      name: "output",
      type: "string",
      description: "The LLM's response to evaluate",
      isOptional: false,
    }
  ]}
/>

## Returns ✅

<PropertiesTable
  content={[
    {
      name: "score",
      type: "number",
      description: "Context relevancy score (0 to scale, default 0-1)",
    },
    {
      name: "info",
      type: "object",
      description: "Object containing the reason for the score",
      properties: [
        {
          type: "string",
          parameters: [
            {
              name: "reason",
              type: "string",
              description: "Detailed explanation of the relevancy assessment",
            }
          ]
        }
      ]
    }
  ]}
/>

## Scoring Details ✅

The metric evaluates how well retrieved context matches the query through binary relevance classification.

### Scoring Process

1. Extracts statements from context:
   - Breaks down context into meaningful units
   - Preserves semantic relationships

2. Evaluates statement relevance:
   - Assesses each statement against query
   - Counts relevant statements
   - Calculates relevance ratio

Final score: `(relevant_statements / total_statements) * scale`

### Score interpretation
(0 to scale, default 0-1)
- 1.0: Perfect relevancy - all retrieved context is relevant
- 0.7-0.9: High relevancy - most context is relevant with few irrelevant pieces
- 0.4-0.6: Moderate relevancy - a mix of relevant and irrelevant context
- 0.1-0.3: Low relevancy - mostly irrelevant context
- 0.0: No relevancy - completely irrelevant context

## Example with Custom Configuration ✅

```typescript
import { openai } from "@ai-sdk/openai";
import { ContextRelevancyMetric } from "@kastrax/evals/llm";

// Configure the model for evaluation
const model = openai("gpt-4o-mini");

const metric = new ContextRelevancyMetric(model, {
  scale: 100, // Use 0-100 scale instead of 0-1
  context: [
    "Basic plan costs $10/month",
    "Pro plan includes advanced features at $30/month",
    "Enterprise plan has custom pricing",
    "Our company was founded in 2020",
    "We have offices worldwide"
  ]
});

const result = await metric.measure(
  "What are our pricing plans?",
  "We offer Basic, Pro, and Enterprise plans.",
);

// Example output:
// {
//   score: 60,
//   info: {
//     reason: "3 out of 5 statements are relevant to pricing plans. The statements about 
//           company founding and office locations are not relevant to the pricing query."
//   }
// }
```

## Related ✅

- [Contextual Recall Metric](./contextual-recall)
- [Context Precision Metric](./context-precision)
- [Context Position Metric](./context-position) 